Bayesian nonlinear regression for large p small n problems

Statistical modelling and inference problems with sample sizes substantially smaller than the number of available covariates are challenging. This is known as large p small n problem. We develop nonlinear regression models in this setup for accurate prediction. In this paper, we introduce a full Bayesian support vector regression model with Vapnik’s 2-insensitive loss function, based on reproducing kernel Hilbert spaces (RKHS). This provides a full probabilistic description of support vector machine (SVM) rather than an algorithm for fitting purposes. We have also considered the relevance vector machine (RVM) introduced by Bishop, Tipping and others. Instead of the original treatment of the RVM relying on the use of type II maximum likelihood estimates of the hyper-parameters, we put a prior on the hyper-parameters and use Markov chain Monte Carlo technique for computation. We apply our model for prediction of blood glucose concentration in diabetics using florescence based optics. We have extended the full Bayesian support vector regression (SVR) and relevance vector regression (RVR) models when the response is multivariate. We have also proposed an empirical Bayes RVM and SVM. The multivariate version of the SVM and RVM is illustrated with a prediction problem in the near-infrared (NIR) spectroscopy. A simulation study is also undertaken to check the prediction accuracy of our models.